Learning to Ride a Bicycle using Iterated Phantom Induction

We build upon our work on iterated phan. ... . .. . -.. . wm IDQUC1O10D anQ Ule worK 01 naDQltZ1V anQ Alstrem OD -applying reinforcemeni learning to the challenging control task of learning to ride a bicycle. Last year phantom induction was demonstrated on a nonlinear but non-dynamica1 task. The bicycle domain is a dynamical system whose next state is a function of the current state and control inputs.. Randl,v and AlstrfIIm demonstrated this task was learnable using Sarsa(~) with shaping. Our approach integrates domain knowledge into the learning process in a flexible way. By taking advantage of what a domain theory expert can easily express, our .system learns to ride. in a few dozen examples rather than the thousands required by the system of Randl,v and AIstr,m. We also show that the two inputs, shifting the rider's center of mass and deflecting the handlebars, are not needed simultaneously for learning. Our system learns to ride using either input alone. By extending phantom induction into the space of dynamical systems we demonstrate the applicability of our learning technique to an important and difficult class of problems.